Selectional preference acquisition through sparse principal component analysis

Words in an utterance are not placed in their respective slots randomly from a uniform distribution. In English, for example, a verb will rarely, if ever, follow a determiner. This is a syntactic restriction. From another perspective, one would not expect to find a word such as defenestration as the object of eat. This is what is known as the selectional preference of a word for another word in terms of its semantic domain. When this selectional preference is restricted to the arguments that a word may take on, it is called its subcategorization frame [11]. In this paper, I lay out a novel, minimally supervised approach to inducing subcategorization frames for verbs using sparse principal component analysis (SPCA)[18], a method which is not based on the traditional singular value decomposition but which utilizes regression techniques. Applying a state-ofthe-art parser for English[4] to a very large corpus[3], I extract all verb-direct object, verb-indirect object, and verb-subject pairings from a large corpus and build verb-by-argument matrices for each type of pairing. After converting the nonzero elements of the matrices to Z-scores and mean centering them, I perform SPCA on these matrices and transform the verbs into the argument space through the principal components. The similarity between the transformed verb and an argument are interpreted to denote the strength of the selectional preference. Therefore, the output is not a hard clustering but an ordered soft clustering that lists arguments for a given verb along with a numeric value for the preference. Previous un-/minimally supervised approaches to semantics in NLP have made use of a technique called latent semantic analysis (LSA). In the following sections, I first discuss LSA and the problems associated with it, then I provide a brief overview of previous approaches to selectional preference acquisition. Then I present sparse PCA, and how it is used in the current experiment. I conclude with an evaluation and discussion of the findings.